Object Detecting Method, Object Detecting Device, And Robot System

ABSTRACT

An object detecting method includes imaging a plurality of target objects with an imaging section and acquiring a first image, recognizing an object position/posture of one of the plurality of target objects based on the first image, counting the number of successfully recognized object positions/postures of the target object, outputting, based on the object position/posture of the target object, a signal for causing a holding section to hold the target object, calculating, as a task evaluation value, a result about whether the target object was successfully held, updating, based on an evaluation indicator including the number of successfully recognized object positions/postures and the task evaluation value, a model for estimating the evaluation indicator from an imaging position/posture of the imaging section and determining an updated imaging position/posture, acquiring a second image in the updated imaging position/posture, and recognizing the object position/posture of the target object based on the second image.

The present application is based on, and claims priority from JPApplication Serial Number 2019-064848, filed Mar. 28, 2019, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an object detecting method, an objectdetecting device, and a robot system.

2. Related Art

When a robot performs work, it is necessary to cause the robot torecognize a position/posture of a target object such as work.

For example, JP A-2013-117795 (Patent Literature 1) discloses aninformation processing device including first acquiring means foracquiring, with a position and/or a posture of a target object set as atarget object state, a distribution of probabilities of the targetobject being in the target object state with respect to target objectstates that the target object can take, second acquiring means foracquiring, from a captured image obtained by the imaging device imagingthe target object having the target object state, a distribution ofsuccess rates of succeeding in identification of the target object withrespect to a predetermined relative target object state for aposition/posture of an imaging device, and determining means fordetermining, based on the distribution of the success rates with respectto the predetermined relative target object state and the distributionof the probabilities for each of a plurality of positions/postures thatthe imaging device can take, a position/posture that the imaging deviceshould take. With such an information processing device, it is possibleto determine a position and a posture of the imaging device that improveidentification accuracy of the target object in an image captured usingthe imaging device. Consequently, it is possible to control a robot armand pick the target object based on the determined position/posture.

However, in the information processing device described in PatentLiterature 1, when a pile of target components are picked,identification accuracy is likely to be deteriorated in a situation inwhich conditions such as illumination for a target object or a state ofpiled target objects changes.

SUMMARY

An object detecting method according to an application example of thepresent disclosure is an object detecting method for detecting aposition/posture of a target object, the object detecting methodincluding: imaging a plurality of the target objects with an imagingsection and acquiring a first image; recognizing the position/posture ofthe target object based on the first image; counting, as a number ofrecognized object positions/postures, a number of successfullyrecognized positions/postures of the target object; outputting, based onthe position/posture of the target object, a signal for causing aholding section to hold the target object; calculating, as a taskevaluation value, a result about whether the target object wassuccessfully held; updating, based on an evaluation indicator includingthe number of recognized object positions/postures and the taskevaluation value, a model for estimating the evaluation indicator froman imaging position/posture of the imaging section and determining anupdated imaging position/posture of the imaging section based on themodel after the update; imaging the plurality of target objects in theupdated imaging position/posture and acquiring a second image; andrecognizing the position/posture of the target object based on thesecond image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram showing a robot system according toa first embodiment.

FIG. 2 is a diagram showing an example of a hardware configuration of anobject detecting device shown in FIG. 1.

FIG. 3 is a flowchart showing an object detecting method according tothe first embodiment.

FIG. 4 is a graph showing a relation between target objects loaded inbulk and an example of an evaluation indicator for determining animaging position/posture.

FIG. 5 is an example of a first image obtained by imaging a state inwhich bolts are used as target objects and the bolts are loaded in bulk.

FIG. 6 is a diagram showing a case in which a bulk state of targetobjects has changed from a bulk state of the target objects shown inFIG. 4.

FIG. 7 is a functional block diagram showing a robot system according toa second embodiment.

FIG. 8 is a functional block diagram showing a robot system according toa third embodiment.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

An object detecting method, an object detecting device, and a robotsystem according to the present disclosure are explained in detail belowbased on embodiments shown in the accompanying drawings.

1. First Embodiment 1.1 Robot System

First, a robot system according to a first embodiment is explained.

FIG. 1 is a functional block diagram showing the robot system accordingto the first embodiment.

In FIG. 1, an X axis, a Y axis, and a Z axis are shown as three axesorthogonal to one another. For convenience of explanation, a distal enddirection of the Z axis is represented as “upper” and a proximal enddirection of the Z axis is represented as “lower”.

A robot system 100 shown in FIG. 1 is used for work such as holding,conveyance, and assembly of a target object 91 (an object) such as anelectronic component. The robot system 100 includes a robot 1 includinga robot arm 10, a camera 3 (an imaging section) having an imagingfunction set in the robot arm 10, an object detecting device 4 thatdetects the target object 91, and a robot control device 5 that controlsdriving of the robot 1 based on a result of the detection by the objectdetecting device 4. The sections are explained in order below.

1.1.1 Robot

The robot 1 shown in FIG. 1 is a so-called six-axis vertical articulatedrobot and includes a base 110 and a robot arm 10 coupled to the base110.

The base 110 is a portion for attaching the robot 1 to any settingplace. In this embodiment, the base 110 is set in a setting place suchas a floor. The setting place of the base 110 is not limited to thefloor or the like and may be, for example, a wall, a ceiling, or amovable truck. Therefore, the Z axis in FIG. 1 is not limited to avertical axis.

The proximal end of the robot arm 10 shown in FIG. 1 is coupled to thebase 110. The robot arm 10 includes an arm 11, an arm 12, an arm 13, anarm 14, an arm 15, and an arm 16. The arms 11 to 16 are coupled in thisorder from the proximal end to the distal end of the robot arm 10. Thearms 11 to 16 are capable of turning with respect to the arms adjacentthereto or the base 110.

The robot 1 includes, although not shown in FIG. 1, a driving devicethat turns the arm 11 with respect to the base 110, a driving devicethat turns the arm 12 with respect to the arm 11, a driving device thatturns the arm 13 with respect to the arm 12, a driving device that turnsthe arm 14 with respect to the arm 13, a driving device that turns thearm 15 with respect to the arm 14, and a driving device that turns thearm 16 with respect to the arm 15. The driving devices include motors,controllers that control driving of the motors, and encoders that detectrotation amounts of the motors. The driving devices are controlledindependently from one another by the robot control device 5.

As shown in FIG. 1, an end effector 17 capable of sucking the targetobject 91 is attached to the distal end of the robot arm 10. The endeffector 17 includes, for example, a gripping hand, a suction hand, anda magnetic hand. The end effector 17 holds the target object 91 placedon a table 92 and performs various kinds of work.

1.1.2 Imaging Section

The camera 3 shown in FIG. 1 is attached to the distal end portion ofthe robot arm 10. An imaging position/posture of the camera 3 shown inFIG. 1 is changed by driving the robot arm 10. The camera 3 can imagethe target object 91 placed on the table 92. The “imagingposition/posture” is, for example, a position/posture in six degrees offreedom for the camera 3.

The camera 3 is communicably coupled to the object detecting device 4.The coupling between the camera 3 and the object detecting device 4 maybe coupling by radio other than coupling by wire.

The camera 3 is one or both of a device capable of acquiringtwo-dimensional images such as a color image, a monochrome image, and aninfrared image of the target object 91 and the periphery of the targetobject 91, that is, a 2D camera and a device capable of acquiring adepth image (surface point group data) of the target object 91 and theperiphery of the target object 91, that is, a 3D camera. Examples of thedevice capable of acquiring the depth image include a three-dimensionalmeasuring device that measures a three-dimensional shape of an imagingtarget with, for example, a phase shift method or an active stereomethod.

1.1.3 Object Detecting Device

The object detecting device 4 is communicably coupled to the camera 3and the robot control device 5. The coupling between the objectdetecting device 4 and the robot control device 5 may be coupling byradio other than coupling by wire.

The object detecting device 4 shown in FIG. 1 includes a camera controlsection 41 (an imaging control section), an object-position/posturecalculating section 42, a recognition evaluating section 43, aholding-position/posture calculating section 44, a task evaluatingsection 45, an imaging-position/posture determining section 46, and adisplay section 47. For example, in order to hold, with the end effector17, a plurality of the target objects loaded in bulk on the table 92,the object detecting device 4 shown in FIG. 1 detects the target object91 and estimates an object position/posture of the target object 91. Theobject detecting device 4 can control the operation of the robot 1 viathe robot control device 5 based on a result of the detection and aresult of the estimation and cause the end effector 17 to hold thetarget object 91. The “object position/posture” is a position/posture insix degrees of freedom for the target object 91 and is, for example, aposition along the X axis, a position along the Y axis, a position alongthe Z axis, a posture about an azimuth angle, a posture about anelevation angle, and a posture about a rotation angle.

The sections of the object detecting device 4 are explained below.

The camera control section 41 shown in FIG. 1 is coupled to the camera3, causes the camera 3 to image the target object 91 placed on the table92, and acquires a first image and a second image. The camera controlsection 41 outputs the acquired first image and the acquired secondimage respectively to the object-position/posture calculating section42. For example, when the camera 3 is configured by both of the 2Dcamera and the 3D camera, the first image and the second image arerespectively formed by two-dimensional images and depth images.

When the camera control section 41 changes the imaging position/postureof the camera 3 based on information concerning an imagingposition/posture output from the imaging-position/posture determiningsection 46, the camera control section 41 outputs a control signal forthe robot arm 10 to the robot control device 5. The camera controlsection 41 controls the robot arm 10 via the robot control device 5 andchanges the imaging position/posture of the camera 3.

The object-position/posture calculating section 42 shown in FIG. 1recognizes an object position/posture of the target object 91 based onthe first image or the second image output from the camera controlsection 41. Specifically, the object-position/posture calculatingsection 42 detects the target object 91 from the first image or thesecond image and performs an arithmetic operation for estimating anobject position/posture of the detected target object 91. Theobject-position/posture calculating section 42 outputs a calculationresult of the object position/posture to the recognition evaluatingsection 43 and the holding-position/posture calculating section 44.

The recognition evaluating section 43 shown in FIG. 1 counts the numberof recognized object positions/postures of the first image based on thecalculation result output from the object-position/posture calculatingsection 42. Specifically, the recognition evaluating section 43 sets, asthe number of recognized object positions/postures, the number of objectpositions/postures of the target object 91 successfully calculated fromthe first image in the object-position/posture calculating section 42.The recognition evaluating section 43 outputs the number of recognizedobject positions/postures to the imaging-position/posture determiningsection 46.

The holding-position/posture calculating section shown in FIG. 1calculates, based on the object position/posture of the target object 91output from the object-position/posture calculating section 42, aholding position/posture of the end effector 17 (a holding section) thatholds the target object 91. The holding-position/posture calculatingsection 44 can calculate the holding position/posture based on adatabase stored for each of types of the target object 91. When thetarget object 91 is sucked and held by, for example, a suction hand, anobject position/posture of a surface (a suction surface) suitable forthe suction only has to be registered in the database in advance.Consequently, the holding-position/posture calculating section 44 cancalculate an object position/posture of the suction surface based on theobject position/posture of the target object 91. Therefore, theholding-position/posture calculating section 44 can calculate a holdingposition/posture of the end effector 17 based on the objectposition/posture of the suction surface. When the target object 91 isgripped by, for example, a gripping hand, an object position/posture ofa surface suitable for the gripping only has to be registered in thedatabase in advance. Consequently, the holding-position/posturecalculating section 44 can calculate a holding position/posture of theend effector 17 suitable for the gripping. The holding-position/posturecalculating section 44 outputs the holding position/posture of the endeffector 17 to the robot control device 5. That is, in order to causethe end effector 17 to grip the target object in the holdingposition/posture, the holding-position/posture calculating section 44outputs a control signal for holding the target object 91 to the robotcontrol device 5. The “holding position/posture” is, for example, aposition/posture in six degrees of freedom for the end effector 17.

The task evaluating section 45 shown in FIG. 1 acquires a result aboutwhether the target object 91 was successfully held by the end effector17 (the holding section), that is, information concerning success orfailure of the holding. The task evaluating section 45 outputs thesuccess or failure of the holding to the imaging-position/posturedetermining section 46 as a task evaluation value. The success orfailure of the holding can be calculated based on a detection result of,for example, a camera that images the end effector 17 or a forcedetector attached to the end effector 17. The camera for confirming thesuccess or failure of the holding may be the same as or may be differentfrom the camera 3 explained above.

The imaging-position/posture determining section shown in FIG. 1calculates an evaluation indicator including the number of recognizedobject positions/postures output from the recognition evaluating section43 and the task evaluation value output from the task evaluating section45. The imaging-position/posture determining section 46 updates, basedon the evaluation indicator, an estimation model for estimating anevaluation indicator from an imaging position/posture and determines animaging position/posture of the camera 3 based on the estimation modelafter the update. The imaging-position/posture determining section 46outputs the determined imaging position/posture to the camera controlsection 41.

The display section 47 shown in FIG. 1 is communicably coupled to theimaging-position/posture determining section 46. The object detectingdevice 4 shown in FIG. 1 includes a display section 47 that displays atleast one of the number of recognized object positions/postures outputfrom the recognition evaluating section 43, the task evaluation valueoutput from the task evaluating section 45, and the evaluation indicatorincluding the number of recognized object positions/postures and thetask evaluation value.

Since the object detecting device 4 includes such a display section 47,for example, it is possible to confirm, as a numerical value,appropriateness of an estimation model in the object detecting device 4.Consequently, it is possible to visually confirm an indicator forquantitatively evaluating soundness of the object detecting device 4.

Examples of the display section 47 include a liquid crystal displaydevice. Information displayed on the display section 47 is not limitedto the information described above and may be other information.

The configuration of the object detecting device 4 according to thefirst embodiment is explained above. The operation of the objectdetecting device 4, that is, an object detecting method is explained indetail below.

FIG. 2 is a diagram showing an example of a hardware configuration ofthe object detecting device 4 shown in FIG. 1.

The object detecting device 4 shown in FIG. 2 includes a processor 4 a,a storing section 4 b, and an external interface 4 c. These componentsare communicably coupled to one another via a system bus 4 d.

The processor 4 a includes a CPU (Central Processing Unit). Theprocessor 4 a reads out and executes various programs and the likestored in the storing section 4 b. Consequently, the processor 4 arealizes various arithmetic operations, various kinds of processing, andthe like in the object detecting device 4.

The storing section 4 b stores various programs and the like executableby the processor 4 a. Examples of the storing section 4 b include avolatile memory such as a RAM (Random Access Memory), a nonvolatilememory such as a ROM (Read Only Memory), and a detachable externalstorage device. Besides the programs, data output from the sectionsexplained above, setting values, and the like are also stored in thestoring section 4 b.

Examples of the external interface 4 c include a wired LAN (Local AreaNetwork) and a wireless LAN.

The functions of the sections of the object detecting device 4 arerealized by the processor 4 a executing the programs. However, at leasta part of the functions may be realized on hardware.

The object detecting device 4 may be disposed in a housing of the robot1, may be disposed outside the housing, or may be provided in a remoteplace via a network or the like.

1.1.4 Robot Control Device

The robot control device 5 has a function of controlling the operationof the robot 1. As shown in FIG. 1, the robot control device 5 iscommunicably coupled to the robot 1 and the object detecting device 4.The robot control device 5 may be coupled respectively to the robot 1and the object detecting device 4 by wire or by radio. A display devicesuch as a monitor, an input device such as a keyboard or a touch panel,and the like may be coupled to the robot control device 5.

Although not shown in FIG. 1, the robot control device 5 includes aprocessor, a storing section, and an external interface. Thesecomponents are communicably coupled to one another via various buses.

The processor includes a processor such as a CPU (Central ProcessingUnit) and executes various programs stored in the storing section.Consequently, it is possible to realize processing such as control ofdriving of the robot 1, various arithmetic operations, anddetermination.

1.2 Object Detecting Method

An object detecting method according to the first embodiment isexplained.

FIG. 3 is a flowchart showing the object detecting method according tothe first embodiment.

The object detecting method shown in FIG. 3 is a method of detecting anobject position/posture of the target object 91. The object detectingmethod according to this embodiment makes it possible to stably detectthe target object 91 even if, for example, a state of the target objectsloaded in bulk changes in a short time. The object detecting methodmakes it possible to control, based on a result of the detection of thetarget object 91, the operation of the robot 1 via the robot controldevice 5 and cause the end effector 17 to stably hold the target object91.

The object detecting method shown in FIG. 3 is a method of detecting anobject position/posture of the target object 91, the method including astep S10 of determining an imaging position/posture of the camera 3, astep S111 of disposing the camera 3 in the imaging position/posture, astep S112 of imaging the plurality of target objects 91 with the camera3 and acquiring a first image, a step S12 of recognizing an objectposition/posture of the target object 91, a step S13 of counting thenumber of recognized object positions/postures and evaluating a resultof the recognition, a step S141 of calculating a holdingposition/posture of the end effector 17, a step S142 of outputting acontrol signal for causing the end effector 17 to hold the target object91, a step S15 of calculating, as a task evaluation value, a resultabout whether the target object 91 was successfully held and evaluatingthe holding result, a step S16 of updating an estimation model based onan evaluation indicator including the number of recognized objectpositions/postures and the task evaluation value and determining anupdated imaging position/posture of the camera 3 based on the estimationmodel after the update, a step S171 of disposing the camera 3 in theupdated imaging position/posture, a step S172 of imaging the targetobject 91 with the camera 3 and acquiring a second image, a step S18 ofrecognizing an object position/posture of the target object 91, and astep S19 of determining whether to finish imaging the target object 91.

With such an object detecting method, it is possible to determine, basedon the fact that the imaging position/posture of the camera 3, whichimages the target object 91, affects success or failure of recognitionof the target object 91 and success or failure of a task, the updateimaging position/posture such that the number of recognitions and thenumber of task successes increase. Moreover, since the updated imagingposition/posture can be sequentially changed during operation, it ispossible to appropriately hold the target object 91 even when aperipheral environment of the target object 91 changes in a short time.Therefore, for example, even in work for holding the target object 91 inan environment in which conditions such as illumination easily change orwork for holding piled target objects 91, it is possible to recognizethe target object 91 without consuming labor and time for tuning forcausing the object detecting device 4 to recognize the target object 91.

The steps are explained below one after another.

1.2.1 Determine an Imaging Position/Posture (Step S10)

First, the imaging-position/posture determining section 46 determines animaging position/posture of the camera 3. Since this step is an initialstate, an image for determining the imaging position/posture is absent.Therefore, in this stage, the imaging-position/posture determiningsection 46 may optionally determine the imaging position/posture.However, in this embodiment, the imaging-position/posture determiningsection 46 determines the imaging position/posture based on theestimation model stored by the imaging-position/posture determiningsection 46. The estimation model is a model for estimating theevaluation indicator from the imaging position/posture of the camera 3.In this step S10, which is a first step, the estimation model does notinclude content based on experiences in the past. Therefore, theimaging-position/posture determining section 46 only has to determinethe imaging position/posture based on an optionally given estimationmodel.

FIG. 4 is a graph showing a relation between the target objects 91loaded in bulk and an example of an evaluation indicator for determiningan imaging position/posture. The imaging position/posture of the camera3 has six degrees of freedom. In FIG. 4, only a position x, which is adegree of freedom of translation along the X axis, is shown. In thefollowing explanation, only the translation along the X axis isexplained. Therefore, although not explained in this embodiment, theobject detecting method is also capable of determining, for theremaining five degrees of freedom, that is, translation along the Yaxis, translation along the Z axis, a posture around the X axis, aposture around the Y axis, and a posture around the Z axis, appropriateimaging positions/postures corresponding to the numbers of recognitionsand the numbers of task successes.

In FIG. 4, a position “a” is shown as an example of the imagingposition/posture determined in this step.

1.2.2 Camera Disposition (Step S111)

Subsequently, the camera control section 41 moves the camera 3 to takethe determined imaging position/posture. That is, the camera controlsection 41 moves the camera 3 such that the position x along the X axisof the camera 3 reaches the position “a”. In this embodiment, the camera3 is attached to the distal end portion of the robot arm 10. The cameracontrol section 41 outputs a control signal to the robot control device5 based on the imaging position/posture output from theimaging-position/posture determining section 46. Consequently, thecamera control section 41 controls the robot arm 10 and moves the camera3 to take a target imaging position/posture.

1.2.3 Target Object Imaging (Step S112)

Subsequently, in the imaging position/posture, the camera 3 captures afirst image to put the plurality of target objects 91 in the same visualfield. The camera control section 41 acquires the captured first image.The camera control section 41 outputs the first image to theobject-position/posture calculating section 42.

1.2.4 Target Object Position/Posture Recognition (Step S12)

Subsequently, the object-position/posture calculating section 42recognizes an object position/posture of the target object 91 based onthe first image. Recognizing an object position/posture of the targetobject 91 means both of detecting the target object 91 in the firstimage and estimating an object position/posture of the target object 91.

Examples of one of specific methods of detecting the target object 91 inthe first image include a method of specifying a contour of the targetobject 91 based on, for example, contrast of a two-dimensional imageincluded in the first image.

Examples of one of specific methods of recognizing an objectposition/posture of the target object 91 include a method of matchingthe first image and design data of the target object 91. The design dataof the target object 91 is, for example, data of three-dimensional CAD(Computer-Aided Design) that can be treated in three-dimensional designdrawing software and data of three-dimensional CG (Computer Graphics)that is configured by constituent elements of a model such as dots,lines, and surfaces and can be treated by three-dimensional computergraphics software.

Examples of another one of the specific methods of recognizing an objectposition/posture of the target object 91 include a method of estimatingan object position/posture of the target object 91 from the first imagewith machine learning using learning data represented by a pair of thefirst image and an object position/posture label. The objectposition/posture label is coordinate data representing the position ofthe target object 91 in the image.

FIG. 5 is an example of the first image obtained by imaging a state inwhich bolts are used as the target objects 91 and loaded in bulk. Linessurrounding the contours of the target objects 91 successfullyrecognized by the object-position/posture calculating section 42 aregiven to the target objects 91.

The first image is, for example, a two-dimensional image or a depthimage and desirably includes both of the two-dimensional image and thedepth image. By acquiring the first image, it is possible to recognize aposition/posture of the target object 91 based on the first image.

When recognizing the object position/posture of the target object 91 inthis way, the object-position/posture calculating section 42 outputs aresult of the recognition, that is, the number of successfullyrecognized object positions/postures to the recognition evaluatingsection 43.

1.2.5 Recognition Result Evaluation (Step S13)

Subsequently, the recognition evaluating section treats, as the numberof recognized object positions/postures, the number of objectpositions/postures of the target object 91 successfully recognized inthe first image. When the number of recognized object positions/posturesis large, the recognition evaluating section 43 can evaluate that animaging position/posture in which the first image is captured is animaging position/posture in which the number of successful recognitionsis large. The recognition evaluating section outputs the number ofrecognized object positions/postures to the imaging-position/posturedetermining section 46.

When recognizing the plurality of target objects 91 in the first image,the recognition evaluating section 43 determines, out of the recognizedplurality of target objects 91, one target object 91 that should be heldby the end effector 17. A criterion for the determination is notparticularly limited. Examples of the criterion include, besides amatching degree of a contour at the time when the estimation result isprojected onto the two-dimensional image as shown in FIG. 5, a matchingdegree of depth at the time when the estimation result is projected ontothe depth image and closeness of the camera 3 and the target object 91.The recognition evaluating section 43 outputs object position/postureinformation of the determined one target object 91 that should be heldby the end effector 17 to the holding-position/posture calculatingsection 44.

1.2.6 End Effector Holding Position/Posture Calculation (Step S141)

Subsequently, the holding-position/posture calculating section 44calculates, based on the object position/posture information of the onetarget object 91 that should be held by the end effector 17, a holdingposition/posture of the end effector 17 that holds the target object 91.For the calculation of a holding position/posture, as explained above, adatabase stored for each of types of the target object 91 is used and aholding position/posture of the end effector 17 optimum for holding thetarget object 91 is calculated based on the database.

1.2.7 Target Object Holding (Step S142)

The holding-position/posture calculating section 44 outputs a controlsignal for causing the end effector 17 to hold the target object 91 inthe holding position/posture explained above. Theholding-position/posture calculating section 44 outputs the controlsignal to the robot control device 5. The robot control device 5controls driving of the robot 1 based on the control signal and causesthe end effector 17 to change the holding position/posture of the endeffector 17. The holding-position/posture calculating section 44attempts to hold the target object 91 with the end effector 17.

1.2.8 Holding Result Evaluation (Step S15)

Subsequently, the task evaluating section 45 acquires, via the robotcontrol device 5, a result about whether the target object 91 wassuccessfully held by the end effector 17, that is, informationconcerning success or failure of the holding. The task evaluatingsection 45 calculates success or failure of the holding as a taskevaluation value and outputs the task evaluation value to theimaging-position/posture determining section 46.

1.2.9 Updated Imaging Position/Posture Determination (Step S16)

Subsequently, the imaging-position/posture determining section 46calculates an evaluation indicator including the number of recognizedobject positions/postures output from the recognition evaluating section43 and the task evaluation value output from the task evaluating section45. The imaging-position/posture determining section 46 reflects thecalculated evaluation indicator on the estimation model stored in theimaging-position/posture determining section 46. The estimation modelis, for example, a model using Bayesian inference. The Bayesianinference has a model based on experiences in the past. The model isupdated by reflecting a most recent evaluation indicator on the model.Consequently, it is possible to determine an optimum updated imagingposition/posture based on the experiences in the past and a most recentevaluation result.

Examples of the evaluation indicator calculated by theimaging-position/posture determining section 46 include a linearcombination of the number of recognized object positions/postures and atask evaluation value indicated by the following expression.

f(x)=D(x)+S(x)

In the expression, f(x) represents an evaluation function representingthe evaluation indicator, D(x) represents the number of recognizedobject positions/postures, and S(x) represents the task evaluationvalue. The task evaluation value is set to be a larger numerical valuewhen the holding by the end effector 17 is successful than when theholding by the end effector 17 is unsuccessful. For example, when thenumber of recognized object positions/postures in one first image isten, the task evaluation value at the time when holding attempted forone target object 91 detected from the first image is successful onlyhas to be set to five and the task evaluation value at the time when theholding is unsuccessful only has to be set to zero. Then, the evaluationfunction f(x) at the time when the holding is successful is 10+5=15. Theevaluation function f(x) at the time when the holding is unsuccessful is10+0=10. In this way, the evaluation indicator including not only thenumber of recognized object positions/postures but also the taskevaluation value is adopted and reflected on the estimation model toupdate the estimation model. Consequently, the estimation model issequentially updated to search for the position x where a large numberof target objects 91 can be recognized and the holding of the targetobject 91 tends to be successful.

By sequentially updating the estimation model, it is possible to causethe estimation model to follow a change in an environment around thetarget object 91. Consequently, by determining an updated imagingposition/posture based on the estimation model, it is possible tocalculate an updated imaging position/posture that can improve a successrate of the holding by the end effector 17.

A value of the task evaluation value is not limited to the valuedescribed above and may be any value.

In the Bayesian inference, the position x where a value of theevaluation function f(x) can be gradually increased is searched byacquiring the evaluation function f(x) for various positions x andreflecting the evaluation function f(x) on the estimation model. Whenthe search is repeated, a correlation curve R1 shown in FIG. 4representing a relation between the position x and the evaluationfunction f(x) is obtained. In FIG. 4, since the evaluation function f(x)is plotted on the vertical axis, a larger value of the evaluationfunction f(x) is represented toward the tip of the vertical axis. Inthis step, for convenience of explanation, it is assumed that, at thispoint in time, the correlation curve R1 is calculated to a certainextent through the search for the position x repeated several times.

Then, in the correlation curve R1 shown in FIG. 4, it can be estimatedthat it is possible to relatively increase the value of the evaluationfunction f(x) by changing the position x of the imaging position/posturefrom a position “a” to a position “b”. Therefore, in this step, theimaging-position/posture determining section 46 determines the position“b” as a new imaging position/posture (an updated imagingposition/posture) and outputs a result of the determination to thecamera control section 41. The new imaging position/posture is, in termsof a probability, an imaging position/posture with a high success rateof the holding by the end effector 17.

1.2.10 Camera Disposition (Step S171)

Subsequently, the imaging-position/posture determining section 46controls the driving of the robot arm 10 of the robot 1 and disposes thecamera 3 to take the determined updated imaging position/posture.

1.2.11 Object Imaging (Step S172)

Subsequently, the camera 3 captures a second image to put the pluralityof target objects 91 in the same visual field. The camera controlsection 41 acquires the captured second image. The camera controlsection 41 outputs the second image to the object-position/posturecalculating section 42.

1.2.12 Target Object Position/Posture Recognition (Step S18)

The object-position/posture calculating section recognizes an objectposition/posture of the target object 91 based on the second image. Amethod of recognizing the target object 91 based on the second image isthe same as the method of recognizing the target object 91 based on thefirst image explained above.

When recognizing the object position/posture of the target object 91 inthis way, the object-position/posture calculating section 42 outputs aresult of the recognition, that is, the number of successfullyrecognized object positions/postures to the recognition evaluatingsection 43.

1.2.13 Determination Concerning whether to Finish Imaging the TargetObject (Step S19)

Subsequently, the recognition evaluating section 43 determines whetherto finish imaging the target object 91. If the holding of all the targetobjects 91 placed on the table 92 is completed, the recognitionevaluating section 43 only has to finish the imaging. On the other hand,if the target object 91 that should be held still remains, therecognition evaluating section 43 returns to step S13 explained above.

The same steps as steps S13, S141, S142, S15, S16, S171, S172, and S18are repeated by the number of the target objects 91. Consequently, it ispossible to hold the target objects 91 one after another and update theestimation model.

The steps performed for the second time are explained as steps S13-2,S14-2 (S141-2 and S142-2), S15-2, S16-2, S17-2 (S171-2 and S172-2), andS18-2.

1.2.14 Recognition Result Evaluation (Step S13-2)

Subsequently, the recognition evaluating section counts, as the numberof recognized object positions/postures, the number of objectpositions/postures of the target object 91 successfully recognized inthe second image.

The recognition evaluating section 43 determines, out of the targetobjects 91 recognized in the second image, one target object 91 thatshould be held by the end effector 17.

1.2.15 End Effector Holding Position/Posture Calculation (Step S141-2)

Subsequently, the holding-position/posture calculating section 44calculates, based on the object position/posture information of the onetarget object 91 that should be held by the end effector 17, a holdingposition/posture of the end effector 17 that holds the target object 91.For the calculation of a holding position/posture, as explained above, adatabase stored for each of types of the target object 91 is used and aholding position/posture of the end effector 17 optimum for holding thetarget object 91 is calculated based on the database.

1.2.16 Target Object Holding (Step S142-2)

The holding-position/posture calculating section 44 outputs a controlsignal for causing the end effector 17 to hold the target object 91 inthe holding position/posture explained above. Theholding-position/posture calculating section 44 outputs the controlsignal to the robot control device 5. The robot control device 5controls driving of the robot arm 10 of the robot 1 based on the controlsignal and causes the end effector 17 to change the holdingposition/posture of the end effector 17. The holding-position/posturecalculating section 44 attempts holding of the target object 91 with theend effector 17.

1.2.17 Holding Result Evaluation (Step S15-2)

Subsequently, the task evaluating section 45 acquires, via the robotcontrol device 5, a result about whether the target object 91 wassuccessfully held by the end effector 17, that is, informationconcerning success or failure of the holding. The task evaluatingsection 45 calculates success or failure of the holding as a taskevaluation value and outputs the task evaluation value to theimaging-position/posture determining section 46.

1.2.18 Updated Imaging Position/Posture Determination (Step S16-2)

Subsequently, the imaging-position/posture determining section 46calculates an evaluation indicator including the number of recognizedobject positions/postures output from the recognition evaluating section43 and the task evaluation value output from the task evaluating section45. The imaging-position/posture determining section 46 reflects thecalculated evaluation indicator on the estimation model stored in theimaging-position/posture determining section 46 and further updates theestimation model. Consequently, it is possible to determine an optimumupdated imaging position/posture based on the estimation model after theupdate.

1.2.19 Camera Disposition (S171-2)

Subsequently, the imaging-position/posture determining section 46controls the driving of the robot arm 10 of the robot 1 and disposes thecamera 3 to take the determined updated imaging position/posture.

1.2.20 Object Imaging (Step S172-2)

Subsequently, the camera 3 captures a third image to put the pluralityof target objects 91 in the same visual field. The camera controlsection 41 acquires the captured third image. The camera control section41 outputs the third image to the object-position/posture calculatingsection 42.

1.2.21 Target Object Holding Position/Posture Recognition (Step S18-2)

Subsequently, the object-position/posture calculating section 42recognizes a holding position/posture of the target object 91 based onthe third image. A method of recognizing the target object 91 based onthe third image is the same as the method of recognizing the targetobject 91 based on the first image explained above.

By further repeating, for the third time, the fourth time, and so on,the same steps as the steps S13-2, S141-2, S142-2, S15-2, S16-2, S171-2,S172-2, and S18-2, which are the steps performed for the second time, itis possible to continue to update the estimation model for searching forthe position x where the value of the evaluation function f(x)increases. As a result, it is possible to continue to search for theposition x where the success rate of holding the target object 91 isfurther increased.

When the estimation model is updated in this way, as shown in FIG. 4,the position x of the imaging position/posture moves to the position“a”, the position “b”, a position “c”, and a position “d”. Every timethe position x moves, a larger value of the evaluation function f(x) isobtained. In the object detecting method according to this embodiment,it is possible to continue to search for an imaging position/posture forincreasing the number of recognized object positions/postures and thetask evaluation value. Therefore, it is possible to efficientlyrecognize and detect, for example, the target object 91 that can beexpected to be held at a high success rate. Consequently, it is possibleto efficiently perform the work for holding the target object 91.

The correction curve R1 changes every time the estimation model isupdated. However, in FIG. 4, for convenience of explanation, it isassumed that the correlation curve R1 does not change. It is notessential that the value of the evaluation function f(x) alwaysincreases every time the estimation model is updated. The value of theevaluation function f(x) may decrease.

Summarizing the above, the object detecting method according to thisembodiment is a method of detecting an object position/posture of thetarget object 91, the method including the step S11 (the steps S111 andS112) of imaging the plurality of target objects 91 with the camera 3(the imaging section) and acquiring a first image, the step S12 ofrecognizing an object position/posture of the target object 91 based onthe first image, the step S13 of counting, as the number of recognizedobject positions/postures, the number of successfully recognized objectpositions/postures of the target object 91, the step S14 (the steps S141and S142) of outputting, based on the object position/posture of thetarget object 91, a control signal for causing the end effector 17 (theholding section) to hold the target object 91, the step S15 ofcalculating, as a task evaluation value, a result about whether thetarget object 91 was successfully held, the step S16 of updating, basedon an evaluation indicator including the number of recognized objectpositions/postures and the task evaluation value, an estimation modelfor estimating the evaluation indicator from an imaging position/postureof the camera 3 and determining an updated imaging position/posture ofthe camera 3 based on the estimation model after the update, the stepsS17 (the steps S171 and S172) of imaging the plurality of target objects91 in the updated imaging position/posture and acquiring a second image,and the step S18 of recognizing the object position/posture of thetarget object 91 based on the second image.

With such an object detecting method, even when a peripheral environmentof the target object 91 changes in a short time, it is possible toappropriately recognize the target object 91 following the change.Therefore, it is possible to cause, based on a result of therecognition, the robot 1 to hold the target object 91 at a high successrate. Consequently, for example, even in work for holding the targetobject 91 in an environment in which conditions such as illuminationeasily change or work for holding the piled target objects 91, it ispossible to recognize the target object 91 without consuming labor andtime for tuning for causing the object detecting device 4 to recognizethe target object 91. As a result, it is possible to efficiently performvarious kinds of work for the target object 91.

In FIG. 6, a bulk state of the target objects 91 has changed from thebulk state of the target objects 91 shown in FIG. 4. Specifically, thetarget objects 91 piled slightly on the left side on the table 92 inFIG. 4 are shifted slightly to the right side in FIG. 6. In such a case,the correlation curve R1 reflecting the bulk state shown in FIG. 4 doesnot reflect the bulk state shown in FIG. 6. Therefore, when it isattempted to hold the target object 91 in the bulk state shown in FIG.6, in order to calculate an imaging position/posture for recognizing thetarget object 91 that can be expected to be held at a high success rate,it is necessary to reflect this change of the bulk state on anestimation model for estimating an evaluation indicator from the imagingposition/posture.

Therefore, in this embodiment, it is desirable to use, as the estimationmodel used for determination of an imaging position/posture, a model ofBayesian inference into which the concept of a forgetting rate isintroduced. The Bayesian inference is an algorithm for calculating anoptimum updated imaging position/posture based on experiences in thepast and a most recent evaluation result as explained above. Byintroducing the concept of the forgetting rate into the Bayesianinference, it is possible to introduce a premise that temporally closerdata is more reliable. Consequently, the estimation model is updatedwhile gradually forgetting experiences in the past. As a result, forexample, about an evaluation indicator obtained in the imagingposition/posture in the position “a” shown in FIG. 6, it is possible toweaken a reflection degree on the estimation model. Then, even if achange from the bulk state of the target objects 91 shown in FIG. 4 tothe bulk state shown in FIG. 6 occurs, by repeating the update of theestimation model, it is possible to gradually eliminate the data of thebulk state shown in FIG. 4 from the estimation model. As a result, it ispossible to gradually construct an estimation model optimized for thebulk state after the change.

When the estimation model optimized for the bulk state shown in FIG. 6is constructed in this way, the correlation curve representing therelation between the position x and the evaluation function f(x) can becalculated as a new correlation curve R2. Based on the correlation curveR2, it can be estimated that a relatively large value of the evaluationfunction f(x) is obtained by setting the position x to, for example, aposition “e” shown in FIG. 6. Consequently, even when the bulk statechanges, it is possible to calculate a new updated imagingposition/posture that can be expected to be held at a high success rate.

As the Bayesian inference into which the concept of the forgetting rateis introduced, for example, nonstationary SBL (Sparse Bayesian Learning)described in “Sparse Bayesian Learning for nonstationary data”,Transactions of the Japanese Society for Artificial Intelligence, volume23, first issue, E (2008), pages 50 to 57 can be used.

The task evaluation value may include an element other than the resultabout whether the target object 91 was successfully held by the endeffector 17 (the holding section). Specifically, the task evaluationvalue may include a result obtained by causing the end effector 17 toperform work using the target object 91 after causing the end effector17 to hold the target object 91. Examples of such work include, when thetarget object 91 is a bolt, work for inserting the bolt into a member,in which a female screw is formed, after holding the bolt with the endeffector 17. Success or failure of such work is incorporated in theevaluation indicator like the success of failure of the holdingexplained above. Consequently, the estimation model updated using theevaluation indicator is updated using the task evaluation valueincluding not only the success or failure of the holding but also thesuccess or failure of the work performed using the held target object91. Then, it is possible to calculate a new updated imagingposition/posture that can be expected to have a high success rate notonly for the success or failure of the holding but also for the successor failure of the work.

When the success or failure of the work is also incorporated in theevaluation indicator, the weight of the evaluation of the success orfailure of the holding and the weight of the evaluation of the successor failure of the work may be differentiated by performing weighting.

The object detecting device 4 according to this embodiment is a devicethat detects an object position/posture of the target object 91, thedevice including the camera control section 41 (the imaging controlsection) that images a first image including the plurality of targetobjects 91 with the camera 3 (the imaging section) and acquires thefirst image, the object-position/posture calculating section 42 thatrecognizes an object position/posture of the target object 91 based onthe first image, the recognition evaluating section 43 that counts, asthe number of recognized object positions/postures, the number ofsuccessfully recognized object positions/postures of the target object91, the holding-position/posture calculating section 44 that calculates,based on the object position/posture of the target object 91, a holdingposition/posture of the end effector 17 (the holding section), whichholds the target object 91, and outputs a control signal for causing theend effector 17 to hold the target object 91 in the holdingposition/posture, the task evaluating section 45 that acquires a resultabout whether the target object 91 was successfully held by the endeffector 17 and calculates a task evaluation value, and theimaging-position/posture determining section 46 that updates, based onan evaluation indicator including the number of recognized objectpositions/postures and the task evaluation value, an estimation modelfor estimating an evaluation indicator from an imaging position/postureof the camera 3 and determines an updated imaging position/posture ofthe camera 3 based on the estimation model after the update. The cameracontrol section 41 causes the camera 3 to capture a second image in theupdated imaging position/posture and acquires the second image. Theobject-position/posture calculating section 42 recognizes the objectposition/posture of the target object 91 based on the second image.

With such an object detecting device 4, since the estimation model forestimating the evaluation indicator from the imaging position/posture issequentially updated, it is possible to calculate the updated imagingposition/posture such that, for example, the target object 91 that canbe expected to be held at a high success rate can be recognized.Therefore, it is possible to realize the robot system 100 having a highsuccess rate of holding by including such the object detecting device 4.

Even when a peripheral environment of the target object 91 changes in ashort time, it is possible to appropriately recognize the target object91 by sequentially updating the estimation model. Consequently, forexample, even in work for holding the target object 91 in an environmentin which conditions such as illumination easily change or work forholding the collapsible piled target objects 91, it is possible torecognize, without consuming labor and time for tuning for causing theobject detecting device 4 to recognize the target object 91, the targetobject 91 that can be expected to be held at a high success rate. As aresult, it is possible to efficiently hold the target object 91 evenunder such an environment.

The robot system 100 according to this embodiment includes the robot 1including the robot arm 10, the camera 3 (the imaging section) set inthe robot arm 10, the object detecting device 4, and the robot controldevice 5 that controls driving of the robot 1 based on a detectionresult of the object detecting device 4.

With such a robot system 100, in the object detecting device 4, evenwhen a peripheral environment of the target object 91 changes in a shorttime, it is possible to appropriately recognize the target object 91 inorder to follow the change. Therefore, it is possible to easily realizethe robot system 100 that can efficiently hold the target object 91.

2. Second Embodiment

A robot system according to a second embodiment is explained.

FIG. 7 is a functional block diagram showing the robot system accordingto the second embodiment.

The robot system according to the second embodiment is explained below.In the following explanation, differences from the robot systemaccording to the first embodiment are mainly explained. Explanationabout similarities is omitted. In FIG. 7, the same components as thecomponents in the first embodiment are denoted by the same referencenumerals and signs.

A robot system 100A according to this embodiment is the same as therobot system 100 according to the first embodiment except that the robotsystem 100A includes an automated guided vehicle 6 mounted with therobot 1.

The automated guided vehicle 6 automatically moves on a predeterminedroute according to guidance by various guidance schemes. In the robotsystem 100A according to this embodiment, the robot 1 is mounted on theautomated guided vehicle 6.

With such a robot system 100A, the object detecting device 4 can alsoappropriately recognize both of target objects 91A and 91B placed inplaces separated from each other. When the places are different, anillumination condition and the like change and visual performance of thetarget objects 91A and 91B by the camera 3 is also different. However,the object detecting device 4 can prevent, by sequentially updating theestimation model, the change in the visual performance from easilyaffecting a success rate of holding. Even when the target objects 91Aand 91B are placed in the places separated from each other, it ispossible to update the estimation model according to the respectiveplaces. As a result, it is possible to suppress deterioration in asuccess rate of holding of the target objects 91A and 91B whileconstructing the robot system 100A having high mobility using theautomated guided vehicle 6.

When the robot 1 is configured such that the position of the robot 1changes, the same estimation model may be continuously updated even ifthe position of the robot 1 changes. However, since peripheralenvironments of the target objects 91A and 91B separated from each otheroften greatly change, visual performance of an image captured by thecamera 3 greatly changes in the robot 1 that holds the target objects91A and 91B. In this case, if an estimation model used in determining animaging position/posture for the target object 91A and an estimationmodel used in determining an imaging position/posture for the targetobject 91B are the same, a deficiency such as divergence of theestimation model due to update is likely to occur. The update of theestimation model is also likely to deteriorate a success rate of holdingto the contrary.

Therefore, the imaging-position/posture determining section 46 desirablyinitializes the estimation model when the position of the robot 1changes. In other words, it is desirable to prepare a plurality ofestimation models independent from one another in advance and switch theestimation models every time the position of the robot 1 changes.Consequently, since the estimation models are optimally updatedaccording to respective environments, even when a robot system is usedto hold (pick) the target objects 91A and 91B while moving like therobot system 100A according to this embodiment, it is possible tosuppress deterioration in the success rate of holding the target objects91A and 91B.

The robot system 100A shown in FIG. 7 includes an automated guidedvehicle control section 48 coupled to the automated guided vehicle 6.The automated guided vehicle control section 48 controls driving of theautomated guided vehicle 6 based on a control signal output from therobot control device 5. The automated guided vehicle control section 48outputs a signal corresponding to the position of the automated guidedvehicle 6 to the object detecting device 4. The object detecting device4 may switch the estimation model based on the signal.

In the second embodiment explained above, the same effects as theeffects in the first embodiment are obtained.

The robot 1 may be mounted on various trucks and the like without beinglimited to the automated guided vehicle 6. The target objects 91A and91B may be placed on a shelf and the like without being limited to thetable 92.

3. Third Embodiment

A robot system according to a third embodiment is explained.

FIG. 8 is a functional block diagram showing the robot system accordingto the third embodiment.

The robot system according to the third embodiment is explained below.In the following explanation, differences from the robot systemaccording to the first embodiment are mainly explained. Explanation ofsimilarities is omitted. In FIG. 8, the same components as thecomponents in the first embodiment are denoted by the same referencenumerals and signs.

A robot system 100B according to this embodiment is the same as therobot system 100 according to the first embodiment except that thecamera 3 (the imaging section) is mounted on a stand 7 away from therobot arm 10 rather than at the distal end portion of the robot arm 10.

The stand 7 is set on a rail or the like laid on a floor on which therobot 1 is set. The camera 3 is fixed to the stand 7 and can image thetarget object 91 placed on the table 92. Although not shown in FIG. 8,the stand 7 moves on the rail based on a control signal output from thecamera control section 41. Consequently, it is possible to dispose thecamera 3 in a target imaging position/posture.

In the robot system 100B including such a fixed camera 3, the sameeffects as the effects of the robot system 100 explained above areobtained.

The object detecting method, the object detecting device, and the robotsystem according to the present disclosure are explained based on theembodiments shown in the figures. However, the present disclosure is notlimited to the embodiments. The components of the sections can bereplaced with any components having the same functions. Any othercomponents may be added to the present disclosure. Further, the robotsystems according to the embodiments are systems including the six-axisvertical articulated robot. The number of axes of the verticalarticulated robot may be five or less or may be seven or more. The robotmay be a horizontal articulated robot instead of the verticalarticulated robot.

What is claimed is:
 1. An object detecting method for detecting anobject position/posture of a target object, the object detecting methodcomprising: imaging a plurality of the target objects with an imagingsection and acquiring a first image; recognizing the objectposition/posture of the target object based on the first image;counting, as a number of recognized object positions/postures, a numberof successfully recognized object positions/postures of the targetobject; outputting, based on the object position/posture of the targetobject, a signal for causing a holding section to hold the targetobject; calculating, as a task evaluation value, a result about whetherthe target object was successfully held; updating, based on anevaluation indicator including the number of recognized objectpositions/postures and the task evaluation value, a model for estimatingthe evaluation indicator from an imaging position/posture of the imagingsection and determining an updated imaging position/posture of theimaging section based on the model after the update; imaging theplurality of target objects in the updated imaging position/posture andacquiring a second image; and recognizing the object position/posture ofthe target object based on the second image.
 2. The object detectingmethod according to claim 1, wherein the first image is atwo-dimensional image or a depth image.
 3. The object detecting methodaccording to claim 1, wherein the model is a model of Bayesian inferenceinto which a concept of a forgetting rate is introduced.
 4. The objectdetecting method according to claim 1, wherein the task evaluation valueincludes a result obtained by causing the holding section to hold thetarget object and perform work using the target object.
 5. An objectdetecting device that detects an object position/posture of a targetobject, the object detecting device comprising: an imaging controlsection configured to capture, with an imaging section, a first imageincluding a plurality of the target objects and acquire the first image;an object-position/posture calculating section configured to recognizethe object position/posture of the target object based on the firstimage; a recognition evaluating section configured to count, as a numberof recognized object positions/postures, a number of successfullyrecognized object positions/postures of the target object; aholding-position/posture calculating section configured to calculate,based on the object position/posture of the target object, a holdingposition/posture of a holding section that holds the target object andoutput a control signal for causing the holding section to hold thetarget object in the holding position/posture; a task evaluating sectionconfigured to acquire a result about whether the target object wassuccessfully held by the holding section and calculate a task evaluationvalue; and an imaging-position/posture determining section configured toupdate, based on an evaluation indicator including the number ofrecognized object positions/postures and the task evaluation value, amodel for estimating the evaluation indicator from an imagingposition/posture of the imaging section and determine an updated imagingposition/posture of the imaging section based on the model after theupdate, wherein the imaging control section causes the imaging sectionto capture a second image in the updated imaging position/posture andacquires the second image, and the object-position/posture calculatingsection recognizes the object position/posture of the target objectbased on the second image.
 6. The object detecting device according toclaim 5, further comprising a display section configured to display atleast one of the number of recognized object positions/postures, thetask evaluation value, and the evaluation indicator.
 7. A robot systemcomprising: a robot including a robot arm; an imaging section set in therobot arm; the object detecting device according to claim 5; and a robotcontrol device configured to control driving of the robot based on adetection result of the object detecting device.
 8. The robot systemaccording to claim 7, further comprising an automated guided vehiclemounted with the robot.
 9. The robot system according to claim 7,wherein the imaging-position/posture determining section initializes themodel when a position of the robot changes.